Automatically Adding Uptimerobot To Gitlab's Monitoring Whitelist With Ansible

GitLab has a number of monitoring endpoints you can ping to make sure that
not only is the rails process running (and not just your webserver) but that
also the background services Gitlab depends on are running successfully too.

I want to monitor my personal Gitlab instance
with UptimeRobot hitting those endpoints for a more accurate uptime
measurement. Gitlab does have a monitoring token I could append to the URL but
that is deprecated. Instead, you should add the monitoring machines to an IP
whitelist. Thankfully, UptimeRobot provides a list of all of their incoming
IP addresses.

Thing is, I’m lazy. I don’t want to have to manually add all of those IP
addresses to the whitelist. I already have the Omnibus GitLab configuration
managed as a template via Ansible, so the best way is to have Ansible
automatically download those lists.

First we’ll set the IP URLs as variables:

uptimerobot_ipv4_url: "https://uptimerobot.com/inc/files/ips/IPv4.txt"
uptimerobot_ipv6_url: "https://uptimerobot.com/inc/files/ips/IPv6.txt"

Then have Ansible download the remote files and register a local variable:

- name: Download the list of IPv4 address of UptimeRobot network
  uri:
    url: "{{ uptimerobot_ipv4_url }}"
    return_content: yes
  register: _ur_ipv4_uri

- name: Download the list of IPv6 address of UptimeRobot network
  uri:
    url: "{{ uptimerobot_ipv6_url }}"
    return_content: yes
  register: _ur_ipv6_uri

Now that we have the two variables, _ur_ipv4_uri and _ur_ipv6_uri, we
can split up the plain text list of IP addresses:

- name: Split up the list of IPs
  set_fact:
    _ur_ipv4_ip_list: "{{ _ur_ipv4_uri.content.split() }}"
    _ur_ipv6_ip_list: "{{ _ur_ipv6_uri.content.split() }}"

Then we merge the two lists into a single one of all of UptimeRobot’s IP
addresses:

- name: Combining the list of IPv4 and IPv6 addresses
  set_fact:
    _ur_all_ips: "{{ _ur_ipv4_ip_list | union(_ur_ipv6_ip_list) }}"

Lastly, here’s the tricky part. I want to include this list of IP addresses
into the monitoring whitelist, while not overriding the current values that
may be included already. Thus, I want to take these two dictionaries:

And merge them together into the sub-key
gitlab_rb.gitlab_rails.monitoring_whitelist. This is done via the lovely
combine filter:

- name: Inject UptimeRobot IP list into monitoring whitelist
  set_fact:
    gitlab_rb: "{{ gitlab_rb | combine({'gitlab_rails': {'monitoring_whitelist': gitlab_rb.gitlab_rails.monitoring_whitelist | default([]) | union(_ur_all_ips) }}, recursive=True) }}"

The important magic happens all in that one line. I take the existing gitlab
configuration dictionary and build a new dictionary that matches it’s
structure. I then take the current monitoring whitelist (which might not
already exist so I default to an empty list) and do a set union with the
UptimeRobot IP addresses. Both dictionaries are passed to the combine filter,
with recursive set to true to not override any other sibling keys in the
original dictionary.

Simple in concept, but Ansible’s defaults don’t make it easy. It took a while
to figure out the appropriate combination of filters to make this merge work
out. Hopefully it’ll save someone else some time.